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Abstract 

J/ip production in top quark rare decays is investigated under the framework 
of NRQCD factorization formalism. Various production channels are studied, 
and we find that the contributions from the color-singlet quark fragmentation 
and the color-octet gluon fragmentation are both over 3 orders larger than that 
from the leading order color-singlet process. The numerical results show that 
the branching ratio B{t c + J /ip + X) is about 10~^^ in the SM, and 10^^° 
in the MSSM. 
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1 



Top quark has been discovered by CDF and DO collaborations at the Femilab Tevatron 
|jl|] and its measured mass is around 175GeV. Because it has a mass of the order of the Fermi 
scale, the top quark may be strongly associated with the electroweak symmetry breaking 
(ESB) sector. So, the studies of top quark (including the production cross section and the 
decay widths etc.) may put forward our knowledge about the ESB mechanism. Furthermore, 
the fact that both Rt and At, remain approximately 2a away from the SM expectations [0 
gives us some hints of the probability that the third generation might couple to some new 
physics (may be associated with the ESB mechanism). All these distinguishing features 
about the top quark and the third generation require us to measure the top quark properties 
precisely. 

The flavor changing neutral current (FCNC) rare decays of heavy quarks have long been 
a subject of intense theoretical and experimental study. The top quark rare decays have also 
been investigated in literatures [3-5]. Because in the SM the branching ratios of the rare 
decay modes are shown to be very small to be observable, {B{t — > cy) ~ 10^^^, B{t cZ) ~ 
10-12 _ iQ-13 ^j^^ ^g-^ _ ^Q-io 0). The FCNC decay modes make an excellent probe 

for models beyond the SM. The Two-Higgs-Doublet models give an enhancement to these 
decay rates by 3 ~ 4 orders of magnitude 0]. The minimal supersymmetric SM (MSSM) also 
gives the same orders enhancement [Q. Recently, some studies of searching for top quark 
rare decays at hadron colliders have also been performed 0. 

In searching for rare decay t eg, a. c quark jet and a gluon jet must be identified, which 
makes it more difficult than the other two modes because of the larger background in the 
hadron collisions. Otherwise, if the c quark or the gluon hadronizes into heavy quarkonium, 
searching for heavy quarkonium in top quark rare decays would further identify the top 
quark effective coupling vertex teg. For this purpose, in this paper we calculate the J/ip 
production rates in top quark rare decays. J/ ip has an extremly clean signature through its 
leptonic decays into e'^e" or fi^fj^~, and this makes it very valuable as trigger for top quark 
rare decays at high energy hadron colliders. Furthermore, on the J/ip production sector, top 
quark rare decays provide another facility to study different production mechanisms. There 
are c quark and gluon in the decay t eg, so we can study the J/ip production via quark 
fragmentation and gluon fragmentation in top quark rare decays. 

In the past few years, a new factorization formalism of heavy quarkonium production in 
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high energy coUisions has been developed by Bodwin, Braaten and Lepage in the context of 
NRQCD 0. In this approach, the production process is factorized into short and long dis- 
tance parts, while the latter is associated with the nonperturbative matrix elements of four- 
fermion operators. Addition to the conventional color-singlet mechanism of heavy quarko- 
nium production, this factorization formalism provides a new mechanism called color-octet 
mechanism, in which the heavy-quark and antiquark pair is produced at short distance in a 
color-octet configuration and subsequently evolves nonperturbatively into physical quarko- 
nium state. After including this new production mechanism, one might explain the ip' [J/ip) 
surplus measured by the CDF group at the Tevatron 0. In the past few years, the applica- 
tions of the NRQCD factorization formalism to J/ipltp') production at various experimental 
facilities have been performed [§. 

In top quark decays, J/ip mainly comes from t W^h followed by h decays h J/ipX 
as well as from direct production t — > hW^g* followed hy g* — > J/ip + X 0. However, 
the J/ip from h decays can be distinguished by using the so-called Second Vertex Detector, 
6-tagging, and the directly produced J/ip can be isolated by studying its energy spectrum 
because it has a very soft energy spectrum 0. As shown in the following calculations, the 
J / ip from top quark rare decays mostly come from parton fragmentation processes in which 
they have hard energy spectrum. So, in top quark rare decays, J/ip is easily detected and 
distinguished, and it therefore can be used to observe the top quark rare decay mode t ^ eg. 

According to the NRQCD factorization formalism, the inclusive production of J/ip in 
top quark rare decays has the following factorized form, 

T(t^c + J/^ + X)=Y.V{t^c+ (cc)[n] + X) {oi'^) . (1) 

n 

Here, f „ is the short- distance coefficient for producing a cc pair in a configuration denoted 
by n (including the angular momentum "^^^"^Lj and the color index 1 or 8). (0"}/^^ is the 
long-distance nonperturbative matrix element demonstrating the probability of the cc pair 
in n configuration evolving into a physical charmonium state J /"(p. f„ can be calculated per- 
turbatively as an expansion in coupling constant a^. Whereas, (Oj^^^^ is a nonperturbative 
parameter, and practically is determined by fitting to the experimental data. The matrix 
element (pi/'^) can be a color-singlet matrix element or a color-octet matrix element due 
to the color index number of the cc pair being 1 or 8. The color-singlet matrix elements 
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are related to the nonrelativistic radial wave function or its derivatives at the origin, but 
the color-octet ones have not the corresponding relations and can only be extracted from 
experiment now. For J/ip production, fortunately the element (O'l^'^ i^Si)^ has been well 
determined from large px prompt J / production at the Tevatron [|^ . 

The leading order color-singlet contributions to J/ ip production in top quark rare decays 
come from the process displayed in Fig. 1(a). In the Feynman diagrams, the teg vertex 
(represented by a black fat dot in Fig.l and Fig. 2) is an effective vertex. In the SM, this 
effective vertex can be calculated by loop calculations and the final result has the following 
form 0, 

K = VciV^iiaip^ + a2q^2 + «37^)^ + {hp^ + &2g2^ + h^^)Rl (2) 

for an internal quark i. Here Vij is KM matrix element, L = (1 — 75)/2, and R = {1 + 75)/2. 
g2, qi, and p are moment of the top quark, c quark and gluon associated with the teg vertex 
respectively. The explicit expressions of the six form factors and hi can be found in Ref. 
0]. Adopting this effective teg vertex in the SM, the decay rates for the leading order color- 
singlet process can be calculated straightforward. The result is lengthy, and it is too tedious 
to write it here. If we take the limit nic <C m^, we will get 

r(-^)(t ^ c + = |^^^^^(a*a3 + blh). (3) 

243 mtrric 

In the numerical calculations, we take the input parameters as 

mt = 175GeV, Mz = 91.2GeV, Mw = SO.lOGeV, (4) 

aem = 1/128.8, as{mt) = 0.10, sin^e^ = 0.23, (5) 

rriu = lOMeV, rris = ISOMeV, m, = 1, 5GeV, nib = 5.0Gev. (6) 
So, the branching ratio for this process is 

5.4X10- (7, 



where, the color-singlet matrix element value (Oi (^Si)) = 1.2GeV^ |T0[ has been used. 



This result is dramatic small, and it is about 8 orders smaller than the branching ratio of 
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the inclusive top quark rare decay t ^ eg which is about 10"^" 0. This is mainly because 
the gluon propagator has a suppression factor of order 0{l/mf) due to the large momentum 
transfer in this process. 

Actually, the dominant color-singlet contributions come from the so-called fragmentation 
processes shown in Fig. 1(b) and Fig. 1(c) for quark and gluon fragmentation respectively. In 
Fig. 1(b), on-shelled c quark (gluon in Fig. 1(c)) is produced from top quark rare decays and 
then fragmentating into the J/ip- The contributions from the fragmentation processes mostly 
come from the region of phase space in which the ip — c [tp — gg in Fig. 1(c)) system has large 
momentum of order rrit and small invariant mass of order ||10| [|ll|]. In the fragmentation 



approximation, the decay rates are given by, for the quark fragmentation |11 



r(t ^ j/ijcg) = r(t 

= T{t 
= T{t 



gc) X Pc^ji^, 
gc) X 
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gc) X 1.2 X 10~^ 



30 



571n2) 



and for gluon fragmentation |TD 



T{t ^ J/^cgg) = T{t 
= T{t 
= T{t 



eg) X P^%/, 
eg) X 8.28 X 10" 
gc) X 3.2 X 10^^ 



4a|(2mc 
mi? 



(9) 



Adding all these contributions Eqs.(^, (^ and (|^) together, we get the total color-singlet 
contributions to J/ip production in top quark rare decays, and the branching ratio is about 



B 



singlet 



{t^c+J/tp + X) = 1.1 X 10 



-14 



(10) 



The dominant color-octet contributions come from the color-octet gluon fragmentation 
process shown in Fig. 2. According to the velocity scaling rules, the color-octet process is 
suppressed by a factor of order v^, but as shown in Fig. 2, the color-octet process has 0{l/al) 
enhancement compared with the color-singlet process in Fig. 1(c). In the fragmentation 
approximation, the decay rates for the color-octet gluon fragmentation process is given by 

m 
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T(t ^ J/^c + X)= T{t ^ eg) X P^%^^ 

= T{t eg) X 1.31 X io-i ^^(y (oi^^i^S, 

ml \ 

= V{t ge) X 1.5 x 10-^ (11) 

and the branching ratio is 

Boctet{t ^ c + J/V^ + X) = 1.5 X 10-l^ (12) 

where the color-octet matrix element (ol^'^i^Si^ = 1.5 x lO^^Ge^^ [0. 

After including the color-singlet and color-octet contributions, the total decay rates for 
J/ip production in top quark rare decays is 

BsM{t-^c+J/i) + X) = 2.QxlQ-^^. (13) 

^From the above results, we can see that the J/ip production in top quark rare decays is 
dominated by the color-singlet quark fragmentation and the color-octet gluon fragmentation, 
and the contributions from these two processes are both 4 orders larger than those from the 
leading order color-singlet process. As shown in Ref . [0] , the energy spectrum of J / ip from 
color-singlet quark fragmentation is hard. The J/ips from color-octet gluon fragmentation 
also have large energies because the t ^ e + J jtp decay in Fig.2 is a two body decay process 
and the energies of the produced J /ips in this process are fixed around one half of the top 
quark mass. That means the J/ips produced in top quark rare decays have much larger 
energies than those from direct production 0. So, the signals of the J/ip produced in top 
quark rare decays are easily to be detected and distinguished. 

In the MSSM, the SUSY QCD correction would provide an enhancement to the top quark 
rare decays The effective teg vertex is given in Ref. [Q, for an internal squark a, 

rv P 

= -i-^K^,KUl^.PLV'' + —PrT''), (14) 

where P = qi + q2, and K^q is the supersymmetric version of KM matrix. The explicit 
expressions of the form factors V and T can be found in Ref. 0. In the MSSM, the mass 
eigenstates of squark are obtained by mixing the left- and right-handed squark with the 
mixing angle 6. Here, we only consider the unmixing case {6 = 0) in which the left-handed 
squark is a mass eigenstate. Our calculations can be easily extended to the mixing case. 
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The mass splitting of the scale top quark and the scale charm quark comes into account, 
which is taken to be rric = 0.9mj. If all scale quark masses would be the same, the decay 
rates would be identical to zero. 

With this effective teg vertex, we can calculate the J/tp production rates in top quark 
rare decays by using the same way discussed above. The decay rates for the leading order 
color-singlet process Fig. 1(a) is 



B^lus'At ^c+J/^ + X) = Bsusvit ^ cg)\P,^j/^ + P^^lj,^ + Pflj/^] (16) 



All the above results are summarized in Fig.3 and Fig. 4. In Fig. 3, we demonstrate the 
dependence of J/ip production branching ratio on the scale top quark mass mi in the case 
that Trig = 150GeV. The dotted line represents the contribution from leading order color- 
singlet process, the dashed line from color-singlet quark and gluon fragmentation processes, 
and the dotted-dashed line from color-octet gluon fragmentation process. In Fig.4, we plot 
the branching ratio as a function of the gluino mass rrig in the case that mj = 2006*6^^. The 
curves in Fig.4 represent the same things as those in Fig.3. iFiom these two figures, we can 
see that similar to that in the SM the J/ip production is dominated by the partons frag- 
mentations (including the color-singlet and color-octet contributions) in the MSSM. Except 
for some lower stop mass region (< ISOGeV^) the branching ratio of B(t c + J/ip + X) 
is not sensitive to the stop quark mass and the gluino mass. Compared with the results 
obtained in the SM Eqs.(|l^), it is shown that the SUSY QCD contributions can enhance 
the J/tp production rates in top quark rare decays by over 4 orders, and the branching ratio 
of B{t ^ c + J/ip + X) is now up to about 10~^°. But it is still too small to be observable. 

In conclusion, in this paper we have calculated J/ip production rates in top quark rare 
decays. We find that both in the SM and in the MSSM the contributions from partons 
fragmentations (color- singlet quark fragmentation and color-octet gluon fragmentation) are 



SUSY 



(LO) 




where V = — and T = T* — T^. Here e is a small number appearing in the SUSY-KM 
matrix, which is taken as = ;| 0]. For the quark and gluon fragmentation (including 
color-singlet and color-octet contributions), the branching ratio is 
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over 3 orders larger than that from leading order color-singlet process. However, the total 
production rates of J/%1) in top quark rare decays are too small to observable, i.e., 10~^^ 
in the SM and 10~^° in the MSSM. So, the new physics effects would be crucial important 
to this decay mode, such as some anomalous top quark interactions. We hope that the 
searching for J/^'s signals in top quark rare decays will be hold at the top quark factories 
such as the LHC and NLC. 
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Figure Captions 



Fig.l. Color-singlet processes in J/'^) production in top quark rare decays, (a) The leading 

order process; (b) quark fragmentation process; (c) gluon fragmentation process. 

Fig. 2. Color-octet gluon fragmentation process in J /ip production in top quark rare decays. 

Fig. 3. The branching ratio B{t ^ c + J/ip + X) as a function of the scale top quark mass in 

the MSSM. The dotted line represents the contribution from the leading order color-singlet 

process, the dashed line from the color-singlet quark and gluon fragmentation processes, and 

the dotted-dashed line from the color-octet gluon fragmentation process. 

Fig.4. The branching ratio B{t — > c + J/t/j + X) as a function of the gluino mass in the 

MSSM. The meanings of the curves are as the same as those in Fig.3. 
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